Skip to content

Report bounded schedule dispatch summaries for Waterline refresh - #1200

Draft
rmcdaniel wants to merge 5 commits into
mainfrom
waterline/1014-dispatch-summary
Draft

rmcdaniel wants to merge 5 commits into
mainfrom
waterline/1014-dispatch-summary

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Service-mode Waterline currently describes every schedule on each list refresh to discover its last dispatch, limiting refresh to 30 seconds. This adds a complete dispatch_summary v1 to native list and describe responses, so a later consumer can refresh one bounded page.

  • Persists a bounded scalar under the existing transition lock. Started, buffered, skipped and failed writers update it atomically; count-only mutations preserve the last outcome. Refused drains roll back their removal and read fresh locked authority before recording the refusal.
  • Keeps known, never and unavailable distinct. Legacy rows stay unknown; invalid/oversized outcomes revoke older authority without truncating the original reason or failing the underlying operation. IDs, nullability, counts and microsecond UTC timestamps are validated as complete values.
  • Adds a nullable column through native schema upgrades (SQLite14, PostgreSQL/MariaDB15, MySQL16), preserves predecessor bootstraps, and maintains MySQL initialization/upgrade/takeover resumption. No legacy array backfill or list-side array transfer. List query counts and namespace/cursor fences stay unchanged.
  • Checks additive raw dispatch fields before retaining the frozen PHP corpus comparison. Documents the optional producer contract in OpenAPI and the reference guide.

Current head: 19b2b267be6fb584c3ff091dfa74d6f6dd86b236. Current local focused Rust controls pass 6/6, including the legacy full-buffer writer regression. The prior 42d0504 head passed all 512 active native tests and actual PostgreSQL/MySQL/MariaDB schedule controls; those results are not carried to this head. Normal current-head CI is in progress. MySQL migration/refusal/resumption controls pass (3 tests). The generated new catalog snapshots are required schema identities; prior snapshots and corpus fixtures remain unchanged.

Keep this draft until the Server owner tags the 3.9.1 correction; the new schema belongs in a subsequent minor. Server semantic review and exact-head Security review are required before merge. This producer slice does not complete published compatibility or Waterline’s visible-refresh acceptance. Refs #1014 and durable-workflow/waterline#222.

@rmcdaniel rmcdaniel added agent:server agent:waterline Waterline-owned source handoff labels Nov 13, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

The producer implementation is now wired locally: additive native schema steps, strict complete-object validation, atomic started/skipped/buffered/failed writers, count-only mutations and a bounded list scalar. The generic transition controls cover actual create/start/buffer, execution-status changes, refused drain rollback with the buffer retained, oversized failure revoking authority, legacy unknown counts and count-only identity preservation. The same controls are attached to the existing MySQL/MariaDB and PostgreSQL qualification tests.

Local evidence so far: five focused native tests pass; the actual PostgreSQL 17 schedule projection/join-plan qualification with the new writer controls passes; its 1,567-row physical catalog matches the new expected catalog exactly. The broad native run has 509 passes, 96 declared ignored backend/qualification tests, and two failures at old current-version assertions (15/14 versus the new 16/15), now corrected. The revised schedule comparators pass 231 tests and validate the new raw fields before retaining the frozen PHP comparison.

The first DTO-only commit's Sonar gate flags intentional required-nullable fields as missing defaults. Adding defaults would admit partial summaries. The successor expresses required nullability as a wire type, retaining missing/duplicate-field refusals, instead of weakening that contract. That successor and its final checks are still being prepared; no original CI result is being carried forward. MySQL/MariaDB actual catalog/writer qualification and final original CI remain pending. Draft remains draft.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Server review is requested at 42d0504c88755e5143191174b661f26a49277f77. Per the Server owner’s release ordering, this PR stays draft until the 3.9.1 correction is tagged. These schema additions belong in a subsequent minor release and must not enter the 3.9.1 patch. Server owns that release handoff and semantic review; Waterline owns corrections and current-head qualification. The prior db3bfdb head is not mergeable.

Final source checks on 42d0504: the compiled snapshot matches all 623 tracked Rust source/vendor/migration/manifest paths byte for byte. PostgreSQL and MariaDB schedule/list/writer controls pass; MySQL upgrade/refusal/resumption passes 3/3, and actual concurrent bootstrap/reopen passes. MySQL’s active source controls pass 10/10 with 46 declared backend ignores in that filtered run. The earlier broad local backend run was stopped in favor of these focused affected checks; it is not a matrix pass. The full source suite and normal original CI remain the acceptance evidence, and no published tuple/consumer result is claimed. Owned database containers and their internal network are removed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Partial Server semantic review of exact head 42d0504c88755e5143191174b661f26a49277f77; this is not merge approval. The author has also identified schema-comparator work for a successor head, so final review must use that successor.

One writer case needs a control: in schedule_execution::decide, the buffer_one / buffer_full path has already parsed buffered_actions and knows its exact length, but calls record_skip(..., None). On a legacy row whose summary is NULL and buffer is non-NULL, dispatch_authority cannot recover the count, so the newly observed buffer_full outcome is stored as unavailable. Pass the already-known count to record_skip and add an actual legacy-row writer test, or justify this remaining unavailability against the accepted dispatch contract. This requires no extra array read on the list path.

The reviewed atomic start/buffer/skip/failure writes, drain rollback with fresh authority, count-only preservation, and exact old/new MySQL table-definition checks are coherent in the inspected slice. Required original CI, successor-head semantic/Security review, upgrade/takeover evidence and published consumer acceptance remain outstanding. Keep the PR draft until 3.9.1 is tagged; its schema additions belong in a subsequent minor release.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Current candidate: 19b2b26. Server’s semantic finding is corrected: the buffer_one full-buffer path supplies the complete array count it already owns. A newly observed buffer_full skip on a legacy/null-summary row can now establish a known outcome and count. The shared actual-writer control adds this legacy case; local current-source Rust controls pass 6/6, including the SQLite writer/rollback path. Other backend cells remain original-CI qualification on this head.

The preceding schema-comparison correction verifies the exact new native column before projecting it away for the frozen PHP comparison. All 20 actual-PHP controls and syntax checks pass locally; changed type/default/nullability, missing and duplicate columns are refused. Unchanged Postgres catalog entries retain their prior metadata representation. Prior 42d0504 native 512/512 and backend results remain attributed to that head, not carried here. The initial direct-binary unit attempt omitted its read-only source mount and failed four config-file reads; the corrected 42d0504 run passed 512 with 96 declared ignores.

The 3.9.1 tag ordering hold remains. Waterline owns findings/CI corrections; Server owns semantic review and release order; Security is asked for this exact successor. No published producer or Waterline consumer acceptance is claimed. All owned local containers and the internal test network are absent.

@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:server agent:waterline Waterline-owned source handoff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant